Contract Pruning #526
Merged
Contract Pruning #526
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…o pj/prunable-data
…o pj/contract-roots
…renterd into pj/contract-spending
ChrisSchinnerl
requested changes
Sep 4, 2023
|
Seems like jape is failing with |
| func (w *worker) PruneContract(ctx context.Context, hostIP string, hostKey types.PublicKey, fcid types.FileContractID, lastKnownRevisionNumber uint64) (deleted, remaining uint64, err error) { | ||
| err = w.withContractLock(ctx, fcid, lockingPriorityPruning, func() error { | ||
| return w.withTransportV2(ctx, hostKey, hostIP, func(t *rhpv2.Transport) error { | ||
| return w.withRevisionV2(ctx, defaultLockTimeout, t, hostKey, fcid, lastKnownRevisionNumber, func(t *rhpv2.Transport, rev rhpv2.ContractRevision, settings rhpv2.HostSettings) (err error) { |
There was a problem hiding this comment.
Can we combine withRevisionV2 and withTransportV2? I think they always go hand-in-hand.
There was a problem hiding this comment.
Ha! I noticed this and it's true in most cases but not all cases (e.g. when forming a contract). Fine to leave as-is imo but we can try and combine them, it works in most cases but I thought it was pushing it a little since we'd have to do a weird exception case for form contract. We also don't always use/need the settings.
ChrisSchinnerl
approved these changes
Sep 6, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a route to the worker that allows pruning deleted sectors from a contract, effectively enabling contract pruning functionality. I've thoroughly tested it on my local node (on testnet) and ran some tests on arequipa. I manually verified contract root diffs before and after as well as perform some upload/download testing while pruning was taking place and believe it all works as expected.
Old hosts are quite (read: really) slow at modifying the storage obligation. For now I've simply lowered the batch size and added a means of timing out the request. The plan is to have the autopilot impose a 5' timeout and nibble away at the prunable data for contracts with v1.5.9 hosts. New hosts are really fast and don't pose an issue at all. Soon the majority of the network will upgrade to v1.6.0 anyway so I don't think it's a problem.
When this PR is merged I'll add a mechanism to automatically prune contracts when the total amount of prunable data exceeds some threshold, as well as add alerts and (ask to) integrate it in the UI perhaps.